Optimal Release of Inventory Using Online Auctions: The Two Item Case

In this paper we analyze policies for optimally disposing inventory using online auctions. We assume a seller has a fixed number of items to sell using a sequence of, possibly overlapping, single-item auctions. The decision the seller must make is when to start each auction. The decision involves a trade-off between a holding cost for each period an item remains unsold, and a higher expected final price the fewer the number of simultaneous auctions underway. Consequently the seller must trade-off the expected marginal gain for the ongoing auctions with the expected marginal cost of the unreleased items by further deferring their release. We formulate the problem as a discrete time Markov Decision Problem and consider two cases. In the first case we assume the auctions are guaranteed to be successful, while in the second case we assume there is a positive probability that an auction receives no bids. The reason for considering these two cases are that they require different analysis. We derive conditions to ensure that the optimal release policy is a control limit policy in the current price of the ongoing auctions, and provide several illustration of results. The paper focuses on the two item case which has sufficient complexity to raise challenging questions

Optimal Strategies in Infinite-state Stochastic Reachability Games

We consider perfect-information reachability stochastic games for 2 players on infinite graphs. We identify a subclass of such games, and prove two interesting properties of it: first, Player Max always has optimal strategies in games from this subclass, and second, these games are strongly determined. The subclass is defined by the property that the set of all values can only have one accumulation point -- 0. Our results nicely mirror recent results for finitely-branching games, where, on the contrary, Player Min always has optimal strategies. However, our proof methods are substantially different, because the roles of the players are not symmetric. We also do not restrict the branching of the games. Finally, we apply our results in the context of recently studied One-Counter stochastic games

Decision Problems for Nash Equilibria in Stochastic Games

We analyse the computational complexity of finding Nash equilibria in stochastic multiplayer games with $\omega$-regular objectives. While the existence of an equilibrium whose payoff falls into a certain interval may be undecidable, we single out several decidable restrictions of the problem. First, restricting the search space to stationary, or pure stationary, equilibria results in problems that are typically contained in PSPACE and NP, respectively. Second, we show that the existence of an equilibrium with a binary payoff (i.e. an equilibrium where each player either wins or loses with probability 1) is decidable. We also establish that the existence of a Nash equilibrium with a certain binary payoff entails the existence of an equilibrium with the same payoff in pure, finite-state strategies.Comment: 22 pages, revised versio

Computing Distances between Probabilistic Automata

We present relaxed notions of simulation and bisimulation on Probabilistic Automata (PA), that allow some error epsilon. When epsilon is zero we retrieve the usual notions of bisimulation and simulation on PAs. We give logical characterisations of these notions by choosing suitable logics which differ from the elementary ones, L with negation and L without negation, by the modal operator. Using flow networks, we show how to compute the relations in PTIME. This allows the definition of an efficiently computable non-discounted distance between the states of a PA. A natural modification of this distance is introduced, to obtain a discounted distance, which weakens the influence of long term transitions. We compare our notions of distance to others previously defined and illustrate our approach on various examples. We also show that our distance is not expansive with respect to process algebra operators. Although L without negation is a suitable logic to characterise epsilon-(bi)simulation on deterministic PAs, it is not for general PAs; interestingly, we prove that it does characterise weaker notions, called a priori epsilon-(bi)simulation, which we prove to be NP-difficult to decide.Comment: In Proceedings QAPL 2011, arXiv:1107.074

Value Iteration for Simple Stochastic Games: Stopping Criterion and Learning Algorithm

Simple stochastic games can be solved by value iteration (VI), which yields a sequence of under-approximations of the value of the game. This sequence is guaranteed to converge to the value only in the limit. Since no stopping criterion is known, this technique does not provide any guarantees on its results. We provide the first stopping criterion for VI on simple stochastic games. It is achieved by additionally computing a convergent sequence of over-approximations of the value, relying on an analysis of the game graph. Consequently, VI becomes an anytime algorithm returning the approximation of the value and the current error bound. As another consequence, we can provide a simulation-based asynchronous VI algorithm, which yields the same guarantees, but without necessarily exploring the whole game graph.Comment: CAV201

Subgame maxmin strategies in zero-sum stochastic games with tolerance levels

We study subgame φ-maxmin strategies in two-player zero-sum stochastic games with finite action spaces and a countable state space. Here φ denotes the tolerance function, a function which assigns a non-negative tolerated error level to every subgame. Subgame φ-maxmin strategies are strategies of the maximizing player that guarantee the lower value in every subgame within the subgame-dependent tolerance level as given by φ. First, we provide necessary and sufficient conditions for a strategy to be a subgame φ-maxmin strategy. As a special case we obtain a characterization for subgame maxmin strategies, i.e. strategies that exactly guarantee the lower value at every subgame. Secondly, we present sufficient conditions for the existence of a subgame φ-maxmin strategy. Finally, we show the possibly surprising result that the existence of subgame φ-maxmin strategies for every positive tolerance function φ is equivalent to the existence of a subgame maxmin strategy

Individual, social, and environmental factors affecting salivary and fecal cortisol levels in captive pied tamarins (Saguinus bicolor)

This is the peer reviewed version of the following article: Price, E., Coleman, R., Ahsmann, J., Glendewar, G., Hunt, J., Smith, T. & Wormell, D. (2019). Individual, social, and environmental factors affecting salivary and fecal cortisol levels in captive pied tamarins (Saguinus bicolor). American Journal of Primatology, 81(8), which has been published in final form at https://doi.org/10.1002/ajp.23033. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-ArchivingPied tamarins (Saguinus bicolor) are endangered New World primates, and in captivity appear to be very susceptible to stress. We measured cortisol in 214 saliva samples from 36 tamarins and in 227 fecal samples from 27 tamarins, and investigated the effects of age, sex, pregnancy, rearing history, social status, weight, group composition, and enclosure type using generalized linear mixed models. There was no effect of age on either fecal or salivary cortisol levels. Female pied tamarins in late pregnancy had higher fecal cortisol levels than those in early pregnancy, or nonpregnant females, but there was no effect of pregnancy on salivary cortisol. Females had higher salivary cortisol levels than males, but there was no effect of rearing history. However, for fecal cortisol, there was an interaction between sex and rearing history. Hand‐reared tamarins overall had higher fecal cortisol levels, but while male parent‐reared tamarins had higher levels than females who were parent‐ reared, the reverse was true for hand‐reared individuals. There was a trend towards lower fecal cortisol levels in subordinate individuals, but no effect of status on salivary cortisol. Fecal but not salivary cortisol levels declined with increasing weight. We found little effect of group composition on cortisol levels in either saliva or feces, suggesting that as long as tamarins are housed socially, the nature of the group is of less importance. However, animals in off‐show enclosures had higher salivary and fecal cortisol levels than individuals housed on‐show. We suggest that large on‐show enclosures with permanent access to off‐exhibit areas may compensate for the effects of visitor disturbance, and a larger number of tamarins of the same species housed close together may explain the higher cortisol levels found in tamarins living in off‐show accommodation, but further research is needed

Markov Decision Processes: Discrete Stochastic Dynamic Programming

The Wiley-Interscience Paperback Series consists of selected books that have been made more accessible to consumers in an effort to increase global appeal and general circulation. With these new unabridged softcover volumes, Wiley hopes to extend the lives of these works by making them available to future generations of statisticians, mathematicians, and scientists. "This text is unique in bringing together so many results hitherto found only in part in other texts and papers. . . . The text is fairly self-contained, inclusive of some basic mathematical results needed, and provides a rich die